(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 
International Bureau 

(43) International Publication Date 
21 June 2001 (21.06.2001) 




PCT 



(10) International Publication Number 

WO 01/43635 Al 



(51) International Patent Classification': A61B 5/04 

(21) International Application Number: PCT/US00/34235 

(22) International Filing Date: 

14 December 2000 (14.12.2000) 



(25) Filing Language: 

(26) Publication Language: 
(30) Priority Data: 



English 
English 



60/170,861 



14 December 1999(14.12.1999) US 



(63) Related by continuation (CON) or continuation-in-part 
(CIP) to earlier application: 

US 60/170,861 (CON) 

Hied on 14 December 1999 (14.12.1999) 

(71) Applicant (for all designated States except US): CALI- 
FORNIA INSTITUTE OF TECHNOLOGY fUS/US]; 
1200 East California Boulevard, Mail Code 201-85, 
Pasadena, CA 91 125 (US). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): PARTHA, Mitra 



[US/US]; 30 B Em Street, Summit, NJ (US). BIJAN, Pe- 
saran [GB/US]; 850 N. Hayworth Avenue, Los Angeles, 
CA 90046 (US). ANDERSON, Richard, A. [US/US]; 
1216 Flintridge Circle, La Canada, CA 9101 1 (US). 

(74) Agent: HARRIS, Scott, C: Fish & Richardson P.C., Suite 
500, 4350 La Jolla Village Drive, San Diego, CA 92122 
(US). 

(81) Designated States (national)*. AE, AG, AL, AM, AT, AU, 
AZ. BA, BB, BG, BR, BY, BZ, CA, CH, CN, CR, CU, CZ, 
DE, DK, DM, DZ, EE, ES, Fl, GB, GD, GE. GH, GM, HR, 
HU, ED, IL, IN, IS, JP, KE, KG. KP, KR, KZ, LC, LK, LR, 
LS. LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, 
NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, 
TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW. 

(84) Designated States (regional)'. ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 
patent (AT, BE, CH, CY, DE, DK, ES, Fl, FR, GB, <5R, IE, 
IT, LU, MC, NL, PT, SE, TR), OAPI patent (BF, BJ, CF, 
CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG). 



Published: 

— With international search report. 



{Continued on next page] 



§S (54) Title: NEURAL PROSTHETIC USING TEMPORAL STRUCTURE IN THE LOCAL FIELD POTENTIAL 

(57) Abstract: Neural prosthetic using temporal 

structure in the local field potential is disclosed. 

In one embodiment, neural activity of a subject 
is measured with an implant (804) in the sensory 
motor cortex (802) of the subject and is used to 
predict an intended movement. The measured 
neural activity may be the local field potential 
(LFP) at an electrode or single unit (SU) activity. 
The spectral structure of the LFP and SU 
activity may be estimated using spectral analysis 
techniques. The estimated LFP and SU responses 
may be used to predict an intended movement by 
the subject. 
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NEURAL PROSTHETIC USING TEMPORAL 
STRUCTURE IN THE LOCAL FIELD POTENTIAL 

This application claims priority to U.S. Provisional 
Patent Application Serial No. 60/170,861, filed on 
5 December 14, 2000, which is herein incorporated by 
reference. 

BACKGROUND 

Many limb prostheses operate in response to muscle 
contractions performed by the user. Some prostheses are 
10 purely mechanical systems. For example, a type of lower 
limb prosthesis operates in response to the motion -of the 
residual limb. When the user's thigh moves forward, 
inertia opens the knee joint of the prosthesis, an 
artificial shin swings forward, and, when the entire 
15 structure locks, the user may pass his or her weight over 
the artificial leg. Other prostheses may incorporate 
electric sensors to measure muscle activity and use the 
measured signals to operate the prosthesis. 

Such prostheses may provide only crude control to 
20 users that have control over some remaining limb 

musculature, and hence may not be useful for patients 
with spinal damage. For these patients, it may be 

1 



WO 01/43635 



PCT/US00/34235 



desirable to measure precursor signals coded for limb 
movement in the patient's brain, and then decode these 
signals to determine the intended movement and/or target. 
The decoded signals could be used to operate a pointing' 
5 device on a computer display screen or robotic 
prosthesis. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a graph illustrating spatial tuning of a 
neuron in a test subject. 
10 Figures 2A and 2B are graphs of waveforms measured 

by an electrode in response to an impending movement. 

Figure 3A is a plot showing the spectral structure 
of single unit (SU) activity. 

Figure 3B is a spectrogram of the spectral structure 
15 shown in Figure 3A. 

Figure A is a two-dimensional plot showing the time- 
frequency coherence between SU and local field potential 
(LFP) spectra. 

Figures 5A and 5B are plots showing the dynamics of 
20 70 Hz power in SU and LFP activity. 

Figures 6A and 6B are plots showing tuning of 70 Hz 
power in LFP and SU activity. 

Figures 7A and 7B are plots showing dynamics of LFP 
power at 70 Hz and 20 Hz. 
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Figure 8 is a schematic diagram of a system 
according to an embodiment. 

Figure 9 illustrates perspective view of an implant 
according to an embodiment. 
5 Figure 10 is a plan view of a subject brain 

illustrating implant sites according to an embodiment. 

Figures 11A and 113 are spectrographs showing the 
spectral structure of LFP signals recorded in a preferred 
and an anti-preferred direction, respectively. 
10 Figure 12 is a plot showing a direction decode. 

Figure 13 is a bar graph showing a decode of 
planning versus execution periods. 

DETAILED DESCRIPTION 

An electrode may be implanted into the cortex of a 
15 subject and used to measure the signals produced by the 
firing of a single unit (SU), i.e., a neuron, in the 
vicinity of an electrode. The SU signal may contain a 
high frequency component. This component may contain 
spikes - distinct events that exceed a threshold value 
20 for a certain amount of time, e.g., a millisecond. 

Spikes may be extracted from the signal and sorted using 
known spike sorting methods. 

Attempts have been made to use the spike trains 
measured from particular neurons to predict a subject's 
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intended movements. The predicted intention could then 
be used to control a prosthetic device. However 
measuring a spike train with a chronic implant and 
decoding an intended movement in real time may be 
5 complicated by several factors. 

In general, measuring SU activity with a chronic 
implant may be difficult because the SU signal may be 
difficult to isolate. An electrode may be in the 
vicinity of more than one neuron, and measuring the 

10 activity of a target neuron may be affected by the 

activity of an adjacent neuron (s). The implant may shift 
position in the patient's cortex after implantation, 
thereby changing the proximity of an electrode to 
recorded neurons over time. Also, the sensitivity of a 

15 chronically implanted electrode to SU activity may 
degrade over time. 

Local field potential (LFP) is an extracellular 
measurement that represents the aggregate activity of a 
population of neurons. The LFP measured at an implanted 

20 electrode during the preparation and execution of a task 
has been found to have a temporal structure that is 
approximately localized in time and space. Information 
provided by the temporal structure of the LFP of neural 
activity appears to correlate to that provided by SU 
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activity, and hence may be used to predict a subject's 
intentions. Unlike SU activity, measuring LFP activity 
does not require isolating the activity of a single unit. 
Accordingly, it may be advantageous to use LFP activity 
5 instead of, or in conjunction with SU activity to predict 
a subject's intended movement in real time. 

Temporal structure is a general term that describes 
patterns in activity over time. Temporal structure 
localized in both time and frequency involves events that 

10 repeat approximately with a period, T, during a time 
interval, after which the period may change. For 
example, the period may get larger, in which case the 
frequency could get smaller. However, for the temporal 
structure to remain localized in frequency as it changes 

15 in time, large changes in the frequency of events cannot 
occur over short intervals in time. 

The relationship between LFP and SU activity in the 
lateral int raparietal (LIP) cortex, responsible for eye 
movement, was studied in an experiment. Neural activity 

20 in LIP was recorded in two adult male Rhesus monkeys 
while performing a memory-saccade task- The neural 
activity was recorded using single tetrodes. 

Each trial of the memory-saccade task began with 
illuminating a central fixation light to which the animal 
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saccaded. The animal was required to maintain fixation 
within a 2° circular window while the fixation light was 
present. After a period of one to two seconds, a target 
light was flashed for 100 milliseconds at one of eight 
5 fixed stimulus locations evenly distributed on a 10° 
circle. Following the target flash, the monkey had to 
remain fixated for a period of one second, at the end of 
which the fixation light was extinguished and the animal 
was required to saacade to the remembered location of the 

10 flashed stimulus. For accurate saccades, the target was 
re-illuminated for a minimum of 500 milliseconds, often 
triggering a corrective saccade, and the animal was 
required re fixate at the new location while the target 
light remained on. Target locations were randomly 

15 interleaved to collect between ten to fifteen successful 
trials for each location in blocked fashion. 

The response of a neuron in the movement planning 
areas of the cortex, for example, the LIP, may be "tuned" 
to a particular direction, that is, the neuron may fire 

20 at a higher rate when a movement is planned in a 

preferred direction than in other directions. For 
example, Figure 1 illustrates the firing frequency 
response 100 of a neuron in the parietal reach region 
(PRR) of a monkey over a 360 degrees of reach directions. 
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The PRR is responsible for the high level planning of 
reaches. This neuron fires at a maximum rate of about 
100 Hz when the monkey plans a reach in the preferred 
direction 102. The direction opposite the preferred 
5 direction is referred to. as the anti-preferred direct ion 
104. 

Recorded neural signals were amplified and filtered 
before being digitized. Digital data was then streamed 
to a disk. Waveforms such as that shown in Figure 2A 
10 were recorded during memory saccade tasks. Figure 2B 

shows a portion 202 of the waveform 200 of Figure 2A on 
an expanded time base. 

SU activity, i.e., spike trains, and the LFP were 
calculated from the recorded neural activity. Both the 
15 SU and LFP data were found to have temporal structure 
that exhibited directional tuning. Furthermore, 
coherency between SU and LFP activity was found to be 
highly significant in a gamma band (25-90 Hz) of the SU 
and LFP activity. Also, single units exhibited phase 
20 locking to the LFP and preferentially fired at the peak 
of an LFP oscillation throughout the memory period. 

Figure 3A shows individual spectra of SU activity in 
the preferred direction for a single ceil from one 
monkey. Figure 3A illustrates the spectrum of SU 
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activity during the memory period 300 and baseline period 
302 from a typical SU . The high frequency limit of the 
spectrum in each case is shown by dotted line 303. 
Ninety-five percent error bars are shown by thin dotted 
5 lines 304-307. 

The spectrum of a Poisson process with the same 
firing rate would fall on the high-frequency limit line 
303. Therefore deviations of the spectrum from this line 
are evidence of temporal structure. 
10 Two significant features are present in the memory 

period spectrum that are absent from the baseline period. 
This indicates the presence of temporal structure during 
working memory. The first significant feature is a 
suppression in power at frequencies below 20 Hz. The 
15 second is a peak in power at a higher frequency band, in 
this case centered at 50 Hz. 

Figure 35 shows a spectrogram 300 for the same unit 
shown in Figure 3A. The spectrogram 300 presents the 
spectrum as a function of time. As shown in Figure 3A, 
20 the spectral structure is sustained throughout the memory 
period, beginning at the initial target illumination and 
continuing though the saccade. 

Figure 4 is a spectrograph 400 that shows the 
coherency between a single cell and the simultaneously 
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recorded LFP averaged across all trials to the preferred 
direction. A sharp increase in the coherence between SU 
activity and the LFP can be seen at 70±20 Kz that exceeds 
99% confidence intervals. This increase is sustained 
5 through the .memory period. When the coherence is 

significant the phase of the coherency is also well- 
organized in this band and has a value of zero radians. 
This is evidence for phase-locking between the SU and the 
LFP during working memory that is sustained throughout 
10 the period. In particular, the phase of the coherency 

indicates that during working memory, the SU fires at the 
peak of broad-band 25-90 Hz LFP activity. In a given 
window, the phase of the coherency is relatively constant 
across frequency indicating that SU and gamma band 
15 activity are synchronous with no time lag between them. 
Since the power in each process is explicitly 
normalized, the increase in coherence during the memory 
period is not related to power increases in either 
process. Instead, it is a result of an increase in the 
20 predictability of one process given the other. This 

indicates that predicting when the single unit will fire 
from LFP activity may be more accurate during working 
memory than during simple fixation. 
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Comparing the power in SU and LFP activity at a 
particular frequency band over time and across trial 
conditions reveals similar organization during the task. 
Figures 5A and 5B show the dynamics of power of neuronal 
5 activity in the 70±20 Hz frequency band. Figure 5A 

illustrates the LFP 500 and SU 502 gamma band activity 
from a single cell, and Figure 5B illustrates the 
population average of LFP 504 and SU 506 activity in one 
monkey . 

10 Figures 6A and 6B compare LF? and SU activity in the 

70110 Hz band in the preferred direction during the task. 
The baseline 600, memory 602, and perisaccade 604 
activity is show against saccade direction. The 
preferred direction is aligned toward the center of the 

15 plot in each case. Elevations in SU activity at 70110 Hz 
are mirrored in the LFP throughout the task, and both 
show strong directional tuning. 

LFP activity between a frequency band 700 at 70110 
Hz and a frequency band 702 at 2015 Hz for trials with 

20 saccades to the preferred directions are compared for a 
single site and a population average in Figures 7A and 
7B, respectively. Activity at 70 Hz is elevated from the 
initial target onset through the saccade. Activity . at 20 
Hz rises toward the end of the memory period and is 
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suppressed peri-saccadically . The elevated 70 Hz 
activity may be related to movement planning. The 
suppression in 20 Hz activity peri-saccadically may be 
related to movement execution. The rise in 20 Hz 
5 activity toward the end of the memory period may reflect 
preparatory aspects of the task and the decrease at the 
end of the memory period may reflect a "go" signal that 
indicates a decision to execute the planned movement. 
Figure S illustrates a system 800 that uses LFP 
10 activity to predict a subject's intended movement plan. 

The activity of neurons in the subject's brain 802 may be 
recorded with an implant 804, such as that shown in 
.Figure 9. The implant 804 may include an array of 
electrodes 900 that measure the action potential (SU) and 
15 extracellular potential (LFP) of ceils in their vicinity. 
In alternate embodiments, the neural activity may be 
measured in forms other than electrical activity. These 
include, for example, optical or chemical changes, or 
changes in blood flew that may be measured by suitable 
20 measuring devices. 

Figure 10 illustrates a number of exemplary implant 
sites. The implant 804 may be implanted at different 
sites for decoding different movements. For example, 
reaches are coded in the parietal reach region (PRR) and 

n 
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area 5 of the superior parietal lobule 1005, saccacies are 
encoded in the LIP 1000, head movements in the ventral 
intraparietal (VIP) cortex 1002, grasps in the anterior 
intraparietal (AIP) cortex 1004, and smooth pursuit eye 
5 movements in the medial superior temporal (MST) cortex 
1006. Reach and other body movements may be encoded in 
other areas of the sensory moior cortex. These areas may 
include, for example, frontal cortical areas involved in 
movement, such as the motor cortex, ventral and dorsal 
0 premotor cortex, supplementary motor area, supplementary 
eye field, frontal eye field, prefrontal cortex, and pre- 
supplementary motor area. 

The PRR 1008 may be a desirable site for a neural 
implant used to decode reaches. The PRR generates high- 
5 level plans for reaching that may require fewer neurons 
to read out the overall goal of a planned movement than 
may be required in other areas. PRR neurons code the 
target of a reach in eye-centered coordinates rather than 
limb-centered coordinates. Also, the PRR neurons code 
o only the goal of the movement, and appear to code only 
the next planned movement in a sequential reach task. 

The PRR may also undergo less reorganization after 
acute nerve damage. The removal of an arm or leg or a 
spinal cord injury may cause grand-scale functional 
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reorganization of cortical maps in both the somatosensory 
and motor cortices. Since the major input to the PRR 
appear to be derived from the visual system, which may 
remain intact in paralyzed patients, the PRR may undergo 
5 less reorganization than the somatosensory and motor 
cortices in these patients. 

Neural activity measured with the implant 804 may be 
amplified in one or more amplifier stages 806 and 
digitized by an analog-to-digital converter (ADC) 808. 

10 In an embodiment, multiple implants may be used. 

Recordings may be made from multiple sites in a brain 
area, with each brain site carrying different 
information, e.g., different directions of planned 
movement. The signals recorded from different implants 

15 may be conveyed on multiple channels. . 

The measured waveform(s), which may include 
frequencies in a range having a lower threshold of about 
1 Hz and an upper threshold of from 5 kHz to 20 kHz may 
be filtered as an analog or digital signal into different 

20 frequency ranges. For example, the waveform may be 

filtered into a low frequency range of say 1-20 Hz, a mid 
frequency range of say 15-200 Hz, which includes the beta 
(15-25 Hz) and gamma (25-90 Hz) frequency bands, and a 

13 
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high frequency range of about 200 Hz to 1 kHz, which may 
include unsorted spike activity. 

In an alternate embodiment, the digitized signal may 
also be input to a spike detector 816 which may detect 
5 and sort spikes using known spike sorting operations. 

The digitized LFP signal, and the sorted spike 
signal if applicable, may be input to a signal processor 
810 for time-frequency localized analysis. 

The digitized LFP and spike (SU) signals may be 
10 represented as spectrograms. For example, Figures 11A 
and 11B illustrate spectrograms 1100, 1102 of LFP 
activity in a preferred and an anti-preferred direction 
obtained in an experiment, and demonstrate the 
directional tuning of the LFP activity. The spectrograms 
15 may be estimated by estimating the spectrum for the data 
in a time window, translating the window a certain 
distance in time, and repeating. Although SU activity is 
a point process composed of discrete events in time 
(action potentials) in contrast to continuous processes 
20 such as the LFP that consist of continuous voltage 

changes, both may be analyzed using similar methods. 

The signal processor 810 may estimate the spectral 
structure cf the digitized LFP and spike signals using 
muititaper methods. Multitaper methods for spectral 
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analysis provide minimum bias and variance estimates of 
spectral quantities, such as power spectrum, which is 
important when the time interval under consideration is 
short . 

5 With multitaper methods, several uncorrelated 

estimates of the spectrum (or cross-spectrum) may be 
obtained from the same section of data by multiplying the 
data by each member of a set of orthogonal tapers. A 
variety of tapers may be used. Such tapers include, for 

10 example, Parzen, Hamming, Hanning, Cosine, etc. 

In an embodiment, the Slepian functions are used. 
The Slepian functions are a family of orthogonal tapers 
given by the prolate spheroidal functions. These 
functions are parameterized by their length in time, T, 

15 and their bandwidth in frequency, W. For choice of T and 
W, up to K=2TW-1 tapers are concentrated in frequency and 
are suitable for use in spectral estimation. 

For an ordinary time series, x t , t=l,...,N. The basic 
quantity for further analysis is the windowed Fourier 

20 transform x { k x \f) : 

xl*\r> = £ w < <*K cxp(-2 OT //) 

1 

where w s {k) ( k~l , 2 , K) are K orthogonal taper 
functions. 

15 



WO 01/43635 



PCT/US00/34235 



For the point process, consider a sequence of event 
times {Tj), j = 1,...,N in the interval [0,T]. The 
quantity for further analysis of point processes is also 
the windowed Fourier transform, denoted by x ( k N) (f) : 

7 = 1 J 

where w 0 (k) is the Fourier transform of the data 
taper at zero frequency and N(T) is the total number of 
spikes in the interval. 

When averaging over trials we introduce an 
10 additional index, I, denoting trail number x ki {f)- 
When dealing with either point or continuous 
process, the multitaper estimates for the spectrum S x (f) , 
cross-spectrum S yx (f), and coherency C w (/) may be given 
by: 

1 A 



is SAf) = -Z\*Af)\ 2 



c.</>- S " (/) 



The auto- and cross-correlation functions may be 
obtained by Fourier transforming the spectrum and cross- 
20 spectrum. 
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In an alternate embodiment the temporal structure of 
the LFP and SU spectral structures may be characterized 
using other spectral analysis methods. For example, 
filters may be combined into a filter bank to capture 
5 temporal structures localized in different frequencies. 
As an alternative to the Fourier transform, a wavelet 
transform may be used to convert the date from the time 
domain into the wavelet domain. Different wavelets, 
corresponding to different tapers, may be used for the 

10 spectral estimation. As an alternative to calculating 

the spectrum on a moving time window, nonstat ionary time- 
frequency methods may be used to estimate the energy of 
the signal for different frequencies at different times 
in one operation. Also, nonlinear techniques such as 

15 artificial neural networks (ANN) techniques may be used 
to learn a solution for the spectral estimation. 

The signal processor 810 may generate a feature 
vector train, for example, a time series of spectra of 
LFP, from the input signals. The feature vector train 

20 may be input to a decoder 812 and operated on to predict 
the subject's intended movement, and from this 
information generate a high level control signal. 

The decoder 812 may use different predictive models 
to predict the intended movement. These may include, for 



CID:<WO 0143635A1 I > 



WO 01/43635 



PCT/US00/34235 



example: probabilistic; Bayesian. decode methods (such 
those described in Zhang, K., Ginzburq, I., McNaughton, 
B.L., and Sejnowski, T.J. (1998), Interpreting Neuronal 
Population Activity by Reconstruction: Unified Framework 
5 with Application to Hippocampal place cells . J 

Neurophysiol 7 8:1017-1044); population vector models 
(such as those described in Lukashin, A.V., Amirikian, 
B.R., and Gecrgopoulos , A. P. (1996). A Simulated Actuator 
Driven by Motor Cortical Signals . Neuroreport 7(15- 
10 17 ): 2597-2 601 ) ; and artificial neural networks. 

The decoder 812. may use a derived transformation 
rule to map a measured neural signal, s, into an action, 
a, for example, a target. Statistical decision theory 
may be used to derive the transformation rule. Factors 
15 in the derivations may include the set of possible neural 
signals, S, and the set of possible actions, A. The 
neuro-motor transform, d, is a mapping for S to A. Other 
factors in the derivation may include the intended target 
9 and a loss function which represents the risk 
20 associated with taking an action, a, when the true 

intention was 0. These variables may be stored in a 
memory device, e.g., a database 814. 

Two approaches may be used to derive the 
transformation rule: A probabilistic approach, involving 
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the intermediate step of evaluating a probabilistic 
relation between s and G and subsequent minimization of 
an expected loss to obtain a neuro-motor transformation; 
and a direct approach, involving direct construction of a 
5 neuro-motor transformation and minimizing the empirical 
loss evaluated over the training set. In terms of so 
called 'neural network' functions, the second approach 
may be regarded as defining a neural network with the 
neural signals as input and the target actions as output, 
10 the weights being adjusted based on training data. In 
both cases, a critical role is played by the loss 
function, which is in some sense arbitrary and reflects 
prior knowledge and biases of the investigator. 

As described above, the measured waveform(s) may ^ e 
15 filtered into a low frequency range of say 1-20 Hz, a mid 
frequency range of say 15-200 Hz, which includes the beta 
(15-25 Hz) and gamma (25-90 Hz) frequency bands, and a 
high frequency range of about 200 Hz to 1 kHz, which may 
include unsorted spike activity. The decoder 812 may 
20 decode an intended direction using the information in the 
gamma frequency band (25-90 Hz) of the LFP spectra and 
the SU spectra. The decoder 812 may decode logical 
signals using information in the gamma (25-90 Hz) and 
beta (15-25 Hz) frequency bands of the LFP spectra and 

CID: <WO ..0143635A1_L> 
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the. SU spectra. The logical information may include a 
decision to execute an action, e.g., a "go" signal. The 
logical information may indicate that the subject is 
entering other states, such as cuing a location, 
5 preparing to execute an action, and scrubbing a planned 
action. 

In an experiment, the decoder decoded a direction 
for a saccade and differentiated the planning and the 
execution periods of the saccade. Figure 12 is a polar 

10 plot of a direction decode 1200 (for one of eight 
directions) using the LFP at a single site in LIP. 
Saccade direction is shown on a circle, and the fraction 
correct 1202 is given by the radius of the point. The 
decode was based on Fisher's linear discriminant trained 

15 with ten trials to each saccade direction for a total of 
eighty trials.' The feature space was three dimensional, 
given by the value of the spectrum estimated on a 500 
millisecond window at frequencies in the gamma frequency 
band. The frequencies that gave the best decode 

20 performance were chosen. As shown in Figure 12, the LFP 
at a single site can predict saccades to the preferred 
direction 1106 and the anti-preferred direction 1208 for 
that site well, and all other directions better than 
chance 1208. 
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Figure 13 is a bar graph representing a decode 1200 
of the planning versus execution periods of the trial 
using the LFP . The decoder had to determine whether the 
daia came from the planning or execution period of the 
5 trial, regardless of saccade direction. The decoder was 
based on Fisher's linear discriminant trained with eighty 
trials from each of the two trial periods for a total of 
160 samples. The feature space was four dimensional 
given by power in the beta and gamma frequency bands. As 
10 shown in Figure 13, the decoder predicted the period 
correctly in over 90% of the samples. 

Once the decoder 812 maps the feature vectors from 
the signal processor 810 to an action, the decoder 812 
may generate a high level signal indicative of the 
15 intended movement and transmit this signal to the device 
controller 820. The device controller 820 may use the 
signal to control the output device 822 ro mimic the 
subject's intended movement. The output device may be, 
for example, a robotic limb, an animated limb or a 
20 pointing device on a display screen, or a functional 
electrical stimulation device implanted into the 
subject's muscles for direct stimulation and control. 

The decoder 812 may need to be recalibrated over 
time. This may be due to inaccuracies in the initial 

21 
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calibration, degradation of the implant to spike activity 
over time, and/or movement of the implant, among other 
reasons . 

In an embodiment, the decoder 812 may use a feedback 
5 controller 824 to monitor the response of the output 
device, compare it to the predicted intended movement, 
and recalibrate the decoder 812 accordingly. The 
feedback controller 824 may include a training program to 
updaie a loss function variable . used by the decoder 812. 

10 Some error may be corrected as the subject learns to 

compensate for the system response based on feedback 
provided by watching the response of the output device. 
The degree of correction due to this feedback response, 
and hence the amount of recaiibrat ion that must be 

15 shouldered by the system 800, may depend in part on the 

degree of plasticity in the region of the brain where the 
implant 804 is positioned. 

A number of embodiments have been described. 
Nevertheless, it will be understood that various 

20 modifications may be made without departing from the 
spirit and scope of the invention. Accordingly, other 
embodiments are within the scope of the following claims. 
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CLAIMS : 

1. A method comprising: 

measuring a local field potential response from 
neural activity in a subject; and 
5 generating a signal indicative of an intended 

movement in response to said measured local field' 
potential response . 

2. The method of claim 1, further comprising: 
cueing a subject to perform a plurality of 

10 movements; 

measuring a local field potential response in 
response to each of said movements; and 

storing the local field potential response 
corresponding to each of said movements in a memory 
15 device. 

3. The method of claim 2, wherein generating the 
signal comprises: 

comparing the measured local field potential 
response to a plurality of the stored local field 
20 potential responses; and 

determining the intended movement in response 
to said comparison. 
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4. The method of claim 1, wherein the signal 
comprises an electric signal. 

5. The method of claim 1, wherein the measuring is 
invasive . 

5 6. The method of claim 1, wherein the local field 

potential response has a temporal structure approximately 
localized in time and frequency. 

7. The method of claim 6, further comprising 
estimating the temporal structure of using spectral 

10 analysis. 

8. The method of claim 1, further comprising 
measuring a spike response having a temporal structure 
approximately localized in time and frequency, and 
wherein the signal is generated .in response to the 

15 measured local field potential response and the measured 
spike response. 

9. The method of claim 1 , wherein the intended 
movement comprises a reach. 

10. The method of claim 9, wherein the reach 
20 comprises a target. 
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11. The method of claim 9, wherein the reach 
comprises a trajectory. 

12. The method of claim 9, wherein the intended 
movement comprises a grasp. 

5 13. The method of claim 1, wherein the intended 

movement comprises a saccade. 

14. The method of claim 1, wherein the intended 
movement comprises a head movement. 

15. The method of claim 1, wherein the intended 
10 movement comprises a smooth pursuit eye movement. 

16. The method of claim 1, wherein the intended 
movement comprises a leg movement. 

17. The method of claim 1, wherein the intended 
movement comprises a go signal . 

15 18. The method of claim 1, wherein the intended 

movement comprises a scrub signal. 

19. The method of claim 1, wherein the neural, 
activity comprises activity in the sensory motor cortex. 
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20. The method of claim 1, wherein the neural 
activity comprises activity in the parietal reach region. 

21. The method of claim 1, wherein the neural 
activity comprises activity in the anterior intrapar ietal 

5 cortex. 

22. The method of claim 1, wherein the neural 
activity comprises activity in the lateral int raparietal 
cortex. 

23. The method of claim 1, wherein the neural 
10 activity comprises activity area 5 of the superior 

parietal lobule. 

24. The method of claim 1, wherein the neural 
activity comprises activity in frontal cortical areas 
involved in movement, said area selected from the motor 

15 cortex, the ventral and dorsal premotor cortex, the 

supplementary motor area, the supplementary eye field, 
the frontal eye field, the prefrontal cortex, and the 
pre-supplementary motor area. 

25. The method of claim 1, wherein the neural 

20 activity comprises activity in the ventral intrapar ietal 
cortex . 
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26. The method of claim 1, wherein the neural 
activity comprises activity in the medial superior 
temporal cortex. 

27. The method of claim 1, further comprising 

5 decoding a direction in a gamma frequency band of the 
local field potential response. 

28. The method of claim 1, further comprising 
decoding a logical signal in at least one of a gamma 
frequency band and a beta frequency band of the local 

10 field potential response. 

29. The method of claim 28, wherein the logical 
signal comprises a "go" signal. 

30. The method of claim 6, wherein the temporal 
structure is represented as a spectrogram. 

15 31. An article comprising: 

a machine-readable medium which stores machine- 
executable instructions, the instructions causing a 
- machine to : 

measure the local field potential response from 
20 neural activity in a subject; and 

27 



CID: <WO 014363SA1_I_> 



WO 01/43635 



PCT/USOO/34235 



generate a signal indicative of an intended 
movement in response to said measured local field 
potential response . 

32. The article of claim 31, wherein the intended 
movement comprises a reach. 

33. An apparatus comprising: 

an input device operative to receive a local 
field potential reading from a measuring device; 

a signal processor coupled to the input device 
and operative to generate a processed signal in response 
to analyzing a temporal structure of the local field 
potential reading; and 

a decoder operative to generate a signal 
indicative of an intended movement in response to said 
processed signal. 

34. The apparatus of claim 33, wherein the 
measuring device comprises an implant. 

35. The apparatus of claim 33, wherein the temporal 
structure is approximately localized in time and 
frequency. 
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35. The apparatus of claim 33, wherein the signal 
processor is operative to analyze the temporal structure 
using spectral analysis. 

36. The apparatus of claim 33, wherein the input 
5 device is further operative to receive a single unit 

activity reading from the measuring device, and wherein 
the signal processor is operative to generate the 
measured response data structure in response to the local 
field potential reading and the single unit activity 
10 reading. 

37. The apparatus of claim 33, wherein the 
generated signal comprises an electric signal. 

38. The apparatus of claim 33, wherein the intended 
movement comprises a reach. 

15 39. The apparatus of claim 38, wherein the reach 

comprises a target. 

40. The apparatus of claim 38, wherein the reach 
comprises a trajectory. 

41. The apparatus of claim 33, wherein the intended 
20 movement comprises a saccade. 
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42. The apparatus of claim 33, wherein the intended 
movement comprises a head movement. 

43. The apparatus of claim 33, wherein the intended 
movement comprises a smooth pursuit eye movement. 

44. The apparatus of claim 33, wherein the intended 
movement comprises a leg movement. 

45. The apparatus of claim 33, wherein the intended 
movement comprises a go signal. 

46. The apparatus of claim 33, wherein the intended 
movement comprises a scrub signal. 

47. The apparatus of claim 33, wherein the decoder 
is operative to. decode a direction from a gamma frequency 
band of the local field potential reading. 

43. The apparatus of claim 33, wherein the decoder 
is operative to decode a logical signal from at least one 
of a gamma frequency band and a beta frequency band of 
the local field potential reading. 

49. A method comprising: 

measuring neural activity; 
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analyzing a time and frequency localized 
representation of the neural activity; and 

generating a signal indicative of an 
intended movement in response to said analysis. 

5 50. The method of claim 49, wherein the neural 

activity comprises measuring a local field potential. 

51. The method of claim 49, wherein the neural 
activity comprises unsorted spiking activity. 

52. The method of claim 49, wherein the neural 
10 activity comprises sorted spiking activity. 

53. The method of claim 49, wherein the 

. measuring comprises measuring an extracellular potential. 
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(57) Abstract: Neural prosthetic using temporal 
structure in the local field potential is disclosed. 
In one embodiment, neural activity ol'a subject is 
measured with an implant (804) in the sensory mo- 
tor cortex (802) of the subject and is used to predict 
an intended movement. The measured neural ac- 
tivity may be the local Held potential (LFP) at an 
electrode or single unit (SU) activity. The spec- 
tral structure of the LFP and SU activity may be 
estimated using spectral analysis techniques. The 
estimated LFP and SU responses may be used to 
predict an intended movement by the subject. 
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NEURAL PROSTHETIC USING TEMPORAL 
STRUCTURE IN THE LOCAL FIELD POTENTIAL 

This application claims priority to U.S. Provisional 
Patent Application Serial No. 60/170,861, filed on 
5 December 14, 2000, which is herein incorporated by 
reference. 

BACKGROUND 

Many limb prostheses operate in response to muscle 
contractions performed by the user. Some prostheses are 
10 purely mechanical systems. For example, a type of lower 
limb prosthesis operates in response to the motion of the 
residual limb. When the user's thigh moves forward, 
inertia opens the knee joint of the prosthesis, an 
artificial shin swings forward, and, when the entire 
15 structure locks, the user may pass his or her weight over 
the artificial leg. other prostheses may incorporate 
electric sensors to measure muscle activity and use the 
measured signals to operate the prosthesis. 

Such prostheses may provide only crude control to 
20 users that have control over some remaining limb 

musculature, and hence may not be useful for patients 
with spinal damage. For these patients, it may be 
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desirable to measure precursor signals coded for limb 
movement in the patient's brain, and then decode these 
signals to determine the intended movement and/or target. 
The decoded signals could be used to operate a pointing 
device on a computer display screen or robotic 
prosthesis. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a graph illustrating spatial tuning of a 
neuron in a test subject. 

Figures 2A and 2B are graphs of waveforms measured 
by an electrode in response to an impending movement . 

Figure 3A is a plot showing the spectral structure 
of single unit (SU) activity. 

Figure 3B is a spectrogram of the spectral structure 
shown in Figure 3A. 

Figure 4 is a two-dimensional plot showing the time- 
frequency coherence between SU and local field potential 
(LFP) spectra. 

Figures 5A and 5B are plots showing the dynamics of 
70 Hz power in SU and LFP activity. 

Figures 6A and 6B are plots showing tuning of 70 Hz 
power in LFP and SU activity. 

Figures 1A and 7B are plots showing dynamics of LFP 
power at 70 Hz and 20 Hz. 
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Figure 8 is a schematic diagram of a system 
according to an embodiment. 

Figure 9 illustrates perspective view of an implant 
according to an embodiment. 
5 Figure 10 is a plan view of a subject brain 

illustrating implant sites according to an embodiment. 

Figures 11A and 11B are spectrographs showing the 
spectral structure of LFP signals recorded in a preferred 
and an anti-preferred direction, respectively. 
10 Figure 12 is a plot showing a direction decode. 

Figure 13 is a bar graph showing a decode of 
planning versus execution periods. 

DETAILED DESCRIPTION 

An electrode may be implanted into the cortex of a 
15 subject and used to measure the signals produced by the 
firing of a single unit (SU) , i.e., a neuron, in the 
vicinity of an electrode. The SU signal may contain a 
high frequency component. This component may contain 
spikes - distinct events that exceed a threshold value 
20 for a certain amount of time, e.g., a millisecond. 

Spikes may be extracted from the signal and sorted using 
known spike sorting methods. 

Attempts have been made to use the spike trains 
measured from particular neurons to predict a subject's 
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intended movements. The predicted intention could then 
be used to control a prosthetic device. However 
measuring a spike train with a chronic implant and 
decoding an intended movement in real time may be 
complicated by several factors. 

In general, measuring SU activity with a chronic 
implant may be difficult because the SU signal may be 
difficult to isolate. An electrode may be in the 
vicinity of more than one neuron, and measuring the 
activity of a target neuron may be affected by the 
activity of an adjacent neuron (s). The implant may shift 
position in the patient's cortex after implantation, 
thereby changing the proximity of an electrode to 
recorded neurons over time. Also, the sensitivity of a 
chronically implanted electrode to SU activity may 
degrade over time. 

Local field potential (LFP) is an extracellular 
measurement that represents the aggregate activity of a 
population of neurons. The LFP measured at an implanted 
electrode during the preparation and execution of a task 
has been found to have a temporal structure that is 
approximately localized in time and space. Information 
provided by the temporal structure of the LFP of neural 
activity appears to correlate to that provided by SU 
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activity, and hence may be used to predict a subject's 
intentions. Unlike SU activity, measuring LFP activity 
does not require isolating the activity of a single unit. 
Accordingly, it may be advantageous to use LFP activity 
instead of, or in conjunction with SU activity to predict 
a subject's intended movement in real time. 

Temporal structure is a general term that describes 
patterns in activity over time. Temporal structure 
localized in both time and frequency involves events that 
repeat approximately with a period, T, during a time 
interval, after which the period may change. For 
example, the period may get larger, in which case the 
frequency could get smaller. However, for the temporal 
structure to remain localized in frequency as it changes 
in time, large changes in the frequency of events cannot 
occur over short intervals in time. 

The relationship between LFP and SU activity in the 
lateral intraparietal (LIP) cortex, responsible for eye 
movement, was studied in an experiment. Neural activity 
in LIP was recorded in two adult male Rhesus monkeys 
while performing a memory-saccade task. The neural 
activity was recorded using single tetrodes. 

Each trial of the memory-saccade task began with 
illuminating a central fixation light to which the animal 
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saccaded. The animal was required to maintain fixation 
within a 2° circular window while the fixation light was 
present. After a period of one to two seconds, a target 
light was flashed for 100 milliseconds at one of eight 
fixed stimulus locations evenly distributed on a 10° 
circle. Following the target flash, the monkey had to 
remain fixated for a period of one second, at the end of 
which the fixation light was extinguished and the animal 
was required to saccade to the remembered location of the 
flashed stimulus. For acqurate saccades, the target was 
re-illuminated for a minimum of 500 milliseconds, often 
triggering a corrective saccade, and the animal was 
required to fixate at the new location while the target 
light remained on. Target locations were randomly 
interleaved to collect between ten to fifteen successful 
trials for each location in blocked fashion. 

The response of a neuron in the movement planning 
areas of the cortex, for example, the LIP, may be "tuned" 
to a particular direction, that is, the neuron may fire 
at a higher rate when a movement is planned in a 
preferred direction than in other directions. For 
example, Figure 1 illustrates the firing frequency 
response 100 of a neuron in the parietal reach region 
(PRR) of a monkey over a 360 degrees of reach directions. 
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The PRR is responsible for the high level planning of 
reaches. This neuron fires at a maximum rate of about 
100 Hz when the monkey plans a reach in the preferred 
direction 102. The direction opposite the preferred 
5 direction is referred to as the anti-preferred direction 
104. 

Recorded neural signals were amplified and filtered 
before being digitized. Digital data was then streamed 
to a disk. Waveforms such as that shown in Figure 2A 

10 were recorded during memory saccade tasks. Figure 2B 

shows a portion 202 of the waveform 200 of Figure 2A on 
an expanded time base. 

SU activity, i.e., spike trains, and the LFP were 
calculated from the recorded neural activity. Both the 

15 SU and LFP data were found to have temporal structure 
that exhibited directional tuning. Furthermore, 
coherency between SU and LFP activity was found to be 
highly significant in a gamma band (25-90 Hz) of the SU 
and LFP activity. Also, single units exhibited phase 

20 locking to the LFP and preferentially fired at the peak 
of an LFP oscillation throughout the memory period. 

Figure 3A shows individual spectra of SU activity in 
the preferred direction for a single cell from one 
monkey. Figure 3A illustrates the spectrum of SU 
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activity during the memory period 300 and baseline period 
302 from a typical SU. The high frequency limit of the 
spectrum in each case is shown by dotted line 303. 
Ninety-five percent error bars are shown by thin dotted 
lines 304-307. 

The spectrum of a Poisson process with the same 
firing rate would fall on the high-frequency limit line 
303. Therefore deviations of the spectrum from this line 
are evidence of temporal structure. 

Two significant features are present in the memory 
period spectrum that are absent from the baseline period. 
This indicates the presence of temporal structure during 
working memory. The first significant feature is a 
suppression in power at frequencies below 20 Hz. The 
second is a peak in power at a higher frequency band, in 
this case centered at 50 Hz. 

Figure 3B shows a spectrogram 300 for the same unit 
shown in Figure 3A. The spectrogram 300 presents the 
spectrum as a function of time. As shown in Figure 3A, 
the spectral structure is sustained throughout the memory 
period, beginning at the initial target illumination and 
continuing though the saccade. 

Figure 4 is a spectrograph 400 that shows the 
coherency between a single cell and the simultaneously 
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recorded LFP averaged across all trials to the preferred 
direction. A sharp increase in the coherence between SU 
activity and the LFP can be seen at 70±20 Hz that exceeds 
99% confidence intervals. This increase is sustained 
5 through the memory period. When the coherence is 

significant the phase of the coherency is also well- 
organized in this band and has a value of zero radians. 
This is evidence for phase-locking between the SU and the 
LFP during working memory that is sustained throughout 

10 the period. In particular , the phase of the coherency 

indicates that during working memory, the SU fires at the 
peak of broad-band 25-90 Hz LFP activity. In a given 
window, the phase of the coherency is relatively constant, 
across frequency indicating that SU and gamma band 

15 activity are synchronous with no time lag between them. 
Since the power in each process is explicitly 
normalized, the increase in coherence during the memory 
period is not related to power increases in either 
process- Instead, it is a result of an increase in the 

20 predictability of one process given the other. This 

indicates that predicting when the single unit will fire 
from LFP activity may be more accurate during working 
memory than during simple fixation. 
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Comparing the power in SU and LFP activity at a 
particular frequency band over time and across trial 
conditions reveals similar organization during the task. 
Figures 5A and 5B show the dynamics of power of neuronal 
activity in the 70120 Hz frequency band. Figure 5A 
illustrates the LFP 500 and SU 502 gamma band activity 
from a single cell, and Figure 5B illustrates the 
population average of LFP 504 and SU 506 activity in one 
monkey. 

Figures 6A and 6B compare LFP and SU activity in the 
70110 Hz band in the preferred direction during the task. 
The baseline 600, memory 602, and perisaccade 604 
activity is show against saccade direction. The 
preferred direction is aligned toward the center of the 
plot in each case. Elevations in SU activity at 70110 Hz 
are mirrored in the LFP throughout the task, and both 
show strong directional tuning. 

LFP activity between a frequency band 700 at 70110 
Hz and a frequency band 702 at 2015 Hz for trials with 
saccades to the preferred directions are compared for a 
single site and a population average in Figures 7A and 
7B, respectively. Activity at 70 Hz is elevated from the 
initial target onset through the saccade. Activity at -20 
Hz rises toward the end of the memory period and is 
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suppressed peri-saccadically. The elevated 70 Hz 
activity may be related to movement planning. The 
suppression in 20 Hz activity peri-saccadically may be 
related to movement execution. The rise in 20 Hz 
activity toward the end of the memory period may reflect 
preparatory aspects of the task and the decrease at the 
end of the memory period may reflect a ^go" signal that 
indicates a decision to execute the planned movement. 

Figure 8 illustrates a system 800 that uses LFP 
activity to predict a subject's intended movement plan. 
The activity of neurons in the subject' s brain 802 may be 
recorded with an implant 804, such as that shown in 
Figure 9. The implant 804 may include an array of 
electrodes 900 that measure the action potential <SU) and 
extracellular potential (LFP) of cells in their vicinity. 
In alternate embodiments, the neural activity may be 
measured in forms other than electrical activity. These 
include, for example, optical or chemical changes, or 
changes in blood flow that may be measured by suitable 
measuring devices. 

Figure 10 illustrates a number of exemplary implant 
sites. The implant 804 may be implanted at different 
sites for decoding different movements- For example, 
reaches are coded in the parietal reach region (PRR) and 
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area 5 of the superior parietal lobule 1005, saccades are 
encoded in the LIP 1000, head movements in the ventral 
intraparietal (VIP) cortex 1002, grasps in the anterior 
intraparietal (AIP) cortex 1004, and smooth pursuit eye 
movements in the medial superior temporal (MST ) cortex 
1006. Reach and other body movements may be encoded in 
other areas of the sensory motor cortex. These areas may 
include, for example, frontal cortical areas involved in 
movement, such as the motor cortex, ventral and dorsal 
premotor cortex, supplementary motor area, supplementary 
eye field, frontal eye field, prefrontal cortex, and pre- 
supplementary motor area. 

The PRR 1008 may be a desirable site for a neural 
implant used to decode reaches. The PRR generates high- 
level plans for reaching that may require fewer neurons 
to read out the overall goal of a planned movement than 
may be required in other areas. PRR neurons code the 
target of a reach in eye-centered coordinates rather than 
limb-centered coordinates. Also, the PRR neurons code 
only the goal of the movement, and appear to code only 
the next planned movement in a sequential reach task. 

The PRR may also undergo less reorganization after 
acute nerve damage. The removal of an arm or leg or a 
spinal cord injury may cause grand-scale functional 
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reorganization of cortical maps in both the somatosensory 
and motor cortices. Since the major input to the PRR 
appear to be derived from the visual system, which may 
remain intact in paralyzed patients, the PRR may undergo 
less reorganization than the somatosensory and motor 
cortices in these patients. 

Neural activity measured with the implant 8 04 may be 
amplified in one or more amplifier stages 806 and 
digitized by an analog-to-digital converter (ADC) 808. 
In an embodiment, multiple implants may be used. 
Recordings may be made from multiple sites in a brain 
area, with each brain site carrying different 
information, e . g ., different directions of planned 
movement. The signals recorded from different implants 
may be conveyed on multiple channels. 

The measured waveform (s), which may include 
frequencies in a range having a lower threshold of about 
1 Hz and an upper threshold of from 5 kHz to 20 kHz may 
be filtered as an analog or digital signal into different 
frequency ranges. For example, the waveform may be 
filtered into a low frequency range of say 1-20 Hz, a mid 
frequency range of say 15-200 Hz, which includes the beta 
(15-25 Hz) and gamma (25-90 Hz) frequency bands, and a 



13 



WO 01/043635 



PCT/USOO/34235 



high frequency range of about 200 Hz to 1 kHz, which may 

include unsorted spike activity. 

In an alternate embodiment, the digitized signal may 

also be input to a spike detector 816 which may detect 

and sort spikes using known spike sorting operations. 
The digitized LFP signal, and the sorted spike 

signal if applicable, may be input to a signal processor 

810 for time-frequency localized analysis. 

The digitized LFP and spike (SU) signals may be 

represented as spectrograms. For example, Figures 11A 
and 11B illustrate spectrograms 1100, 1102 of LFP 
activity in a preferred and an anti-preferred direction 
obtained in an experiment, and demonstrate the 
directional tuning of the LFP activity. The spectrograms 
may be estimated by estimating the spectrum for the data 
in a time window, translating the window a certain 
distance in time, and repeating. Although SU activity is 
a point process composed of discrete events in time 

(action potentials) in contrast to continuous processes 
such as the LFP that consist of continuous voltage 
changes, both may be analyzed using similar methods. 

The signal processor 810 may estimate the spectral 
structure of the digitized LFP and spike signals using 
multitaper methods. Multitaper methods for spectral 
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analysis provide minimum bias and variance estimates of 
spectral quantities, such as power spectrum, which is 
important when the time interval under consideration is 
short . 

5 With multitaper methods, several uncorrelated 

estimates of the spectrum (or cross-spectrum) may be 
obtained from the same section of data by multiplying the 
data by each member of a set of orthogonal tapers. A 
variety of tapers may be used. Such tapers include, for 

10 example, Parzen, Hamming, Manning, Cosine, etc. 

In an embodiment, the Slepian functions are used. 
The Slepian functions are a family of orthogonal tapers 
given by the prolate spheroidal functions. These 
functions are parameterized by their length in time, T, 

15 and their bandwidth in frequency, W. For choice of T and 
W, up to K=2TW-1 tapers are concentrated in frequency and 
are suitable for use in spectral estimation. 

For an ordinary time series, x t , t=l,...,lSi. The basic 
quantity for further analysis is the windowed Fourier 

20 transform xj; x) (f) : 

i 

where w t (k) (k=l, 2, K) are K orthogonal taper 
functions. 
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For the point process, consider a sequence of event 
times { r } } , j = 1,„.,N in the interval [0,T]. The 
quantity for further analysis of point processes is also 
the windowed Fourier transform, denoted by x£ N) (f) : 

where w 0 (k) is the Fourier transform of the data 

taper at zero frequency and N(T) is the total number of 
spikes in the interval. 

When averaging over trials we introduce an 
10 additional index, I, denoting trail number x ki (f) . 

When dealing with either point or continuous 
process, the multitaper estimates for the spectrum S x (f) , 
cross-spectrum S yx (f) , and coherency C^(/) may be given 
by: 



is " s x u-) = ±±\x k (f)\ 2 



The auto- and cross-correlation functions may be 
obtained by Fourier transforming the spectrum and cross- 
20 spectrum. 
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In an alternate embodiment the temporal structure of 
the LFP and SU spectral structures may be characterized 
using other spectral analysis methods. For example, 
filters may be combined into a filter bank to capture 

5 temporal structures localized in different frequencies. 
As an alternative to the Fourier transform, a wavelet 
transform may be used to convert the date from the time 
domain into the wavelet domain. Different wavelets, 
corresponding to different tapers, may be used for the 

10 spectral estimation. As an alternative to calculating 

the spectrum on a moving time window, nonstationary time- 
frequency methods may be used to estimate the energy of 
the signal for different frequencies at different times 
in one operation. Also, nonlinear techniques such as 

15 artificial neural networks (ANN) techniques may be used 
to learn a solution for the spectral estimation. 

The signal processor 310 may generate a feature 
vector train, for example, a time series of spectra of 
LFP, from the input signals. The feature vector train 

20 may be input to a decoder 812 and operated on to predict 
the subject's intended movement, and from this 
information generate a high level control signal. 

The decoder 812 may use different predictive models 
to predict the intended movement. These may include, for 
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example: probabilistic; Bayesian decode methods (such 
those described in Zhang, K., Ginzburg, I., McNaughton, 
B.L., and Sejnowski, T.J. (1998), Interpreting Neuronal 
Population Activity by Reconstruction: Unified Framework 
with Application to Hippocampal place cells . J 
Neurophysiol 79:1017-104 4); population vector models 
(such as those described in Lukashin, A.V., Amirikian, 
B.R., and Georgopoulos , A, P. (1996). A Simulated Actuator 
Driven by Motor Cortical Signals . Neuroreport 7(15- 
17) : 2597-2601) ; and artificial neural networks. 

The decoder 812 may use a derived transformation 
rule to map a measured neural signal, s, into an action, 
a, for example, a target. Statistical decision theory 
may be used to derive the transformation rule. Factors 
in the derivations may include the set of possible neural 
signals, S, and the set of possible actions, A. The 
neuro-motor transform, d, is a mapping for S to A. Other 
factors in the derivation may include the intended target 
0 and a loss function which represents the risk 
associated with taking an action, a, when the true 
intention was 0. These variables may be stored in a 
memory device, e.g., a database 814. 

Two approaches may be used to derive the 
transformation rule: A probabilistic approach, involving 
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the intermediate step of evaluating a probabilistic 
relation between s and 9 and subsequent minimization of 
an expected loss to obtain a neuro-motor transformation; 
and a direct approach, involving direct construction of a 
neuro-motor transformation and minimizing the empirical 
loss evaluated over the training set. In terms of so 
called 'neural network' functions, the second approach 
may be regarded as defining a neural network with the 
neural signals as input and the target actions as output, 
the weights being adjusted based on training data. In 
both cases, a critical role is played by the loss 
function, which is in some sense arbitrary and reflects 
prior knowledge and biases of the investigator. 

As described above, the measured waveform (s) may be 
filtered into a low frequency range of say 1-20 Hz, a mid 
frequency range of say 15-200 Hz, which includes the beta 
(15-25 Hz) and gamma (25-90 Hz) frequency bands, and a 
high frequency range of about 200 Hz to 1 kHz, which may 
include unsorted spike activity. The decoder 812 may 
decode an intended direction using the information in the 
gamma frequency band (25-90 Hz) of the LFP spectra and 
the SU spectra. The decoder 812 may decode loqical 
signals using information in the gamma (25-90 Kz) and 
beta (15-25 Hz) frequency bands of the LFP spectra and 
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the SU spectra. The logical information may include a 
decision to execute an action, e.g., a ^go" signal. The 
logical information may indicate that the subject is 
entering other states, such as cuing a location, 
preparing to execute an action, and scrubbing a planned 
action. 

In an experiment, the decoder decoded a direction 
for a saccade and differentiated the planning and the 
execution periods of the saccade. Figure 12 is a polar 
plot of a direction decode 1200 (for one of eight 
directions) using the LFP at a single site in LIP- 
Saccade direction is shown on a circle, and the fraction 
correct 1202 is given by the radius of the point. The 
decode was based on Fisher' s linear discriminant trained 
with ten trials to each saccade direction for a total of 
eighty trials. The feature space was three dimensional, 
given by the value of the spectrum estimated on a 500 
millisecond window at frequencies in the gamma frequency 
band. The frequencies that gave the best decode 
performance were chosen. As shown in Figure 12, the LFP 
at a single site can predict saccades to the preferred 
direction 11*06 and the anti-preferred direction 1208 for 
that site well, and all other directions better than 
chance 1208. 
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Figure 13 is a bar graph representing a decode 1200 
of the planning versus execution periods of the trial 
using the LFP. The decoder had to determine whether the 
data came from the planning or execution period of the 
5 trial, regardless of saccade direction. The decoder was 
based on Fisher's linear discriminant trained with eighty 
trials from each of the two trial periods for a total of 
160 samples. The feature space was four dimensional 
given by power in the beta and gamma frequency bands. As 
10 shown in Figure 13, the decoder predicted the period 
correctly in over 90% of the samples. 

Once the decoder 812 maps the feature vectors from 
the signal processor 810 to an action, the decoder 812 
may generate a high level signal indicative of the 
15 intended movement and transmit this signal to the device 
controller 820. The device controller 820 may use the 
signal to control the output device 822 to mimic the 
•subject's intended movement . The output device may be, 
for example, a robotic limb, an animated limb or a 
20 pointing device on a display screen, or a functional 
electrical stimulation device implanted into the 
subject's muscles for direct stimulation and control. 

The decoder 812 may need to be recalibrated over 
time. This may be due to inaccuracies in the initial 
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calibration, degradation of the implant to spike activity 
over time, and/or movement of the implant, among other 
reasons . 

In an embodiment, the decoder 812 may use a feedback 
controller 824 to monitor the response of the output 
device, compare it to the predicted intended movement, 
and recalibrate the decoder 812 accordingly. The 
feedback controller 824 may include a training program to 
update a loss function variable used by the decoder 812. 

Some error may be corrected as the subject learns to 
compensate for the system response based on feedback 
provided by watching the response of the output device. 
The degree of correction due to this feedback response, 
and hence the amount of recalibration that must be 
shouldered by the system 800, may depend in part on the 
degree of plasticity in the region of the brain where the 
implant 804 is positioned. 

A number of embodiments have been described. 
Nevertheless, it will be understood that various 
modifications may be made without departing from the 
spirit and scope of the invention. Accordingly, other 
embodiments are within the scope of the following claims. 
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CIAIMS: 

1. A method comprising: 

measuring a local field potential response from 
neural activity in a subject; and 
5 generating a signal indicative of an intended 

movement in response to said measured local field 
potential response . 

2. The method of claim 1, further comprising: 
cueing a subject to perform a plurality of 

10 movements; 

measuring a local field potential response in 
response to each of said movements; and 

storing the local field potential response 
corresponding to each of said movements in a memory 
15 device. 

3. The method of claim 2, wherein generating the 
signal comprises: 

comparing the measured local field potential 
response to a plurality of the stored local field 
20 potential responses; and 

determining the intended movement in response 
to said comparison. 
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4. 



The method of claim 1, wherein the signal 



comprises an electric signal. 



5. 



The method of claim 1, wherein the measuring is 



invasive . 



6. 



The method of claim 1, wherein the local field 



potential response has a temporal structure approximately 



localized in time and frequency. 



7. 



The method of claim 6, further comprising 



estimating the temporal structure of using spectral 
analysis. 



measuring a spike response having a temporal structure 
approximately localized in time and frequency, and 
wherein the signal is generated in response to the 
measured local field potential response and the measured 
spike response. 

9. The method of claim 1, wherein the intended 
movement comprises a reach. 

10. The method of claim 9, wherein the reach 
comprises a target. 



8. 



The method of claim 1, further comprising 
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11. The method of claim 9, wherein the reach 
comprises a trajectory. 

12. The method of claim 9, wherein the intended 
movement comprises a grasp. 

5 13. The method of claim 1, wherein the intended 

movement comprises a saccade. 

14. The method of claim 1, wherein the intended 
movement comprises a head movement. 

15. The method of claim 1, wherein the intended 
10 movement comprises a smooth pursuit eye movement. 

16. The method of claim 1, wherein the intended 
movement comprises a leg movement. 

17. The method of claim 1, wherein the intended 
movement comprises a go signal. 

15 18. The method of claim 1, wherein the intended 

movement comprises a scrub signal. 

19. The method of claim 1, wherein the neural 

activity comprises activity in the sensory motor cortex, 
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20. The method of claim 1, wherein the neural 
activity comprises activity in the parietal reach region. 

21. The method of claim 1, wherein the neural 
activity comprises activity in the anterior intraparietal 

5 cortex. 

22. The method of claim 1, wherein the neural 
activity comprises activity in the lateral intraparietal 
cortex. 

23. The method of claim 1, wherein the neural 
10 activity comprises activity area 5 of the superior 

parietal lobule. 

24. The method of claim 1, wherein the neural 
activity comprises activity in frontal cortical areas 
involved in movement, said area selected from the motor 

15 cortex, the ventral and dorsal premotor cortex, the 

supplementary motor area, the supplementary eye field, 
the frontal eye field, the prefrontal cortex, and the 
pre-supplementary motor area. 

25. The method of claim 1, wherein the neural 

20 activity comprises activity in the ventral intraparietal 
cortex . 
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26. The method of claim 1, wherein the neural 
activity comprises activity in the medial superior 
temporal cortex. 

27. The method of claim 1, further comprising 

5 decoding a direction in a gamma frequency band of the 
local field potential response. 

28. The method of claim 1, further comprising 
decoding a logical signal in at least one of a gamma 
frequency band and a beta frequency band of the local 

10 field potential response. 

29. The method of claim 28, wherein the logical 
signal comprises a "go" signal. 

30. The method of claim 6, wherein the temporal 
structure is represented as a spectrogram. 

15 31. An article comprising: 

a machine-readable medium which stores machine- 
executable instructions, the instructions causing a 
machine to: 

measure the local field potential response from 
20 neural activity in a subject; and 
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generate a signal indicative of an intended 
movement in response to said measured local field, 
potential response. 

32. The article of claim 31, wherein the intended 
movement comprises a reach. 

33. An apparatus comprising: 

an input device operative to receive a local 
field potential reading from a measuring device; 

a signal processor coupled to the input: device 
and operative to generate a processed signal in response 
to analyzing a temporal structure of the local f ±eld 
potential reading; and 

a decoder operative to generate a signal 
indicative of an intended movement in response to said 
processed signal. 

34. The apparatus of claim 33, wherein the 
measuring device comprises an implant. 

35. The apparatus of. claim 33, wherein the temporal 
structure is approximately localized in time and 
frequency. 
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35. The apparatus of claim 33, wherein the signal 
processor is operative to analyze the temporal structure 
using spectral analysis . 

36. The apparatus of claim 33, wherein the input 
5 device is further operative to receive a single unit 

activity reading from the measuring device, and wherein 
the signal processor is operative to generate the 
measured response data structure in response to the local 
field potential reading and the single unit activity 
10 reading. 

37. The apparatus of claim 33, wherein the 
generated signal comprises an electric signal. 

38. The apparatus of claim 33, wherein the intended 
movement comprises a reach. 

15 39. The apparatus of claim 38, wherein the reach 

comprises a target. 

40. The apparatus of claim 38, wherein the reach 
comprises a trajectory. 

41. The apparatus of claim 33, wherein the intended 
20 movement comprises a saccade. 
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42. The apparatus of claim 33, wherein the intended 
movement comprises a head movement. 

43. The apparatus of claim 33, wherein the intended 
movement comprises a smooth pursuit eye movement. 

44. The apparatus of claim 33, wherein the intended 
movement comprises a leg movement. 

45. The apparatus of claim 33, wherein the intended 
movement comprises a go signal. 

46. The apparatus of claim 33, wherein the intended 
movement comprises a scrub signal. 

47. The apparatus of claim 33, wherein the decoder 
is operative to decode a direction from a gamma frequency 
band of the local field potential reading. 

48. The apparatus of claim 33, wherein the decoder 
is operative to decode a logical signal from at least one 
of a gamma frequency band and a beta frequency band of 
the local field potential reading. 

49. A method comprising: 

measuring neural activity; 
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analyzing a time and frequency localized 
representation of the neural activity; and 

generating a signal indicative of an 
intended movement in response to said analysis. 

5 50. The method of claim 4 9, wherein the neural 

activity comprises measuring a local field potential. 

51. The method of claim 4 9, wherein the neural 
activity comprises unsorted spiking activity. 

52. The method of claim 49, wherein the neural 
10 activity comprises sorted spiking activity. 

53. The method of claim 4 9, wherein the 
measuring comprises measuring an extracellular potential. 
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